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1 . Introduction . 

The purpose of this report is to develop several analytical 
(probabilistic) computer system models in a form that allows them to be 
"fitted" to actual measurement data. Some of these models have been 
validated by comparison with runs of actual sample programs; in general, 
the agreement achieved has been reasonably good, and suggestions are made 
here of ways in which the models may be improved so that agreement is likely 
to be better. Further measurements are being made, and the success with which 
they can be described by models will be reported later. 

The measurement data referred to has been taken on a Honeywell 6000- 
series computer at the Joint Technical Support Activity (JTSA) , a field 
activity of the Defense Communications Agency in Res ton, Virginia. The 
measurements were made by Mr. Barry Wallack of the JTSA, to whom I am grateful 
for enthusiastic and careful cooperation. 



2. Basic Assumptions * 



The mathematical assumption that pervades our analysis is that our 
systems act as if they are Markovian . That is, at any instant of time, t, 
we can describe the system state by a random vector (N^ (t ) ,N^ (t ) , . . . ,N^(t ) ) 

= N^(t) ; furthermore, knowledge of the value of £J(t) allows us to calculate 
all future event probabilities. Concretely, N^(t) may denote the number of 
programs awaiting and undergoing CPU processing at t, while N^(t) ,N^(t) , . . . 
represent the number at each of several different memory devices (e.g. disk 
units or channels thereto, tape units, etc.). The Markovian assumption made 
is equivalent to assuming that independently and exponentially distributed 
"burst times" on CPU and memory devices occur. This assumption cannot be 
rigorously true, but derived results are often insensitive to this assumption 
(see formulas for the number present in steady state in the infinite server 
queue; reference [8]). Furthermore, the usual measurements of computer system 
performance only allow identification of a single rate parameter for a device. 
Experiments with realistic workload material will be used to check the validity 
of this assumption as it reflects upon system performance characteristics of 
interest , such as device utilization, and system throughput. 

The reason for making the Markovian assumption is, in the final analysis, 
to obtain simple, easily understood expressions that involve a few basic 
parameters. Sensitivity of system performance to departures from this type of 
assumption can sometimes be effectively assessed by use of another approach, 
that of diffusion approximation analysis. 
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3 . Simple Configurations . 



Actual computer systems, such as the Honeywell 6000 and the IBM 360, 
must be represented as complex arrays of serving points and queues. Before 
proceeding to complex models, however, I will describe some much simpler ones. 
The implications of these can be easily compared to the results of running 
simple but realistic program material through an actual system. Such valida- 
tion attempts are currently in progress at JTSA, conducted by Mr. B. Wallack. 

Configuration 1: One Processor, One-Channel to Disc. 

Suppose one processor (CPU) services J programs that occupy core 
Assume that the reason that a continuous period of CPU activity on a particular 
program (a "CPU burst" for short) terminates is the requirement for information 
contained on a disc. The latter is accessed through a single channel. Because 
of the presence of other programs in the system, there may well be queueing 
for the channel. After this queueing is completed, the program accesses the 
disc, reads out the information, and, at the completion of the 10 activity, 
assumes a place in the CPU queue. The process continues until all computation 
is completed and the program exits, to be instantly replaced by another. Note 
that disc mount requests may, in the IBM system, delay the acceptance of new 
jobs into core. This effect will be modeled later; its effect is not present 
in the usual simple cyclic models for multiprogramming systems. 

Several simplifying assumptions are notable: 

(a) there are a fixed number, J, of programs in core, 

(b) computation and disc activity do not overlap, i.e. I do not here admit 
the possibility (which apparently exists) that CPU and disc activity 
can simultaneously occur on the same program, 

(c) operating system requirements on the CPU (overhead) are not explicitly 
included , 
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(d) Programs are homogeneous enough to be described statistically in a 
simple fashion. 

We allow for these assumptions when the model is compared to actual computer 
runs of simple program material. 

Our configuration appears as below: 



O O O 



O (CPU) h 






y 

^ Channel 



Configuration 1. 



Analysis . A Markov model of this system requires the specification of the 

rate parameters of CPU and Channel-Disc: denote the expected length of a 

-1 -1 
CPU burst by A , and of a Disk burst (actual read time from Disc) by y 

Furthermore, the system state can be taken to be N(t), the number of programs 

at the CPU state — waiting plus being served. Obviously J-N(t) are then 

present at the Disc stage. It may then be shown (see [3], Gaver and Thompson) 

that (i) there is a long-run or steady-state probability distribution for 



N(t): 



where P{ 



} 



p = lim P{N(t)=n} 
n 

t-x» 



denotes a probability measure. 

P n ' P 0© ; 



Pr> = 






i- ffl 



M'l J+l 



Furthermore 
0 £ n J 



(3.1) 



(3.2) 



Discussion . Several points can be made. 
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(a) The value of p ^ is the long-run fraction of time there are exactly n 
(n=0, or 1, or_ 2, ... ojr J) programs at the CPU stage. If X > h then 
the jobs tend to be IQ bound . If core size is increased, meaning J 
is increased, then CPU idleness decreases to a positive lower bound: 

J = 1, CPU idleness = p n = -~— 

0 X+y 

J = 00 , CPU idleness = Pq = 1- ~>0 

If X < y, meaning the jobs tend to be CPU bound , then CPU idleness 
decreases, but now to zero: 

J = 1, CPU idleness = p^ = 

0 X+y 

J = °°, CPU idleness = p^ = 0 . 



(b) The system productivity may be assessed as follows. Let program i 

require b^ CPU bursts, each of expected length X ^ . Then the expected 
time to finish k programs is 




(3.3) 



The expected amount of CPU time available in a time period 
mately) equal to T(l-p^) . Thus T must be such that 

I(i-P 0 ) - X j, »i 

1=1 



or 



T 



is (approxi- 



(3.4) 



T = 



X (1" 



p o } 



k 

I 

i=l 



If the number of bursts per program is thought of as random with mean 
then 



(3.5) 

b 



5 



T = 



kf-1 

1 A J (1-P 0 ) 



= (Number of Programs to be Processed) * (Mean CPU Time per Program) 

(CPU utilization) 



The expression (3.6) is increasingly accurate as k becomes large and 
as program durations become more homogeneous. 

(c) Since p^ depends only upon the ratio and since the number of CPU 

bursts nearly equals the number of CPU bursts for each program 

Mean CPU Time per Program _ b/A _ y_ . 

Mean Disc Time per Program E/y A 

The left-hand side of (3.6) can be estimated by running a set of represent- 
ative programs through an actual system and utilizing a monitor. This 
has been carried out at JTSA on the Honeywell 6000. Since actual CPU 
activity includes overhead, and the latter is not explicitly modeled 
(see Lewis and Shedler, [6], for an explicit model in the IBM context) > the 
rate A must be reduced to represent overhead. 



(d) 



The expected number of programs at the CPU stage is, in the long run, 
given by the formula 



E[N] = lim E [N (t) ] 

t-x» 




Of course 

E [number at Channel] = J - E [number at CPU] 



= J - E [N] . 
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Configuration 2: One Processor, Two Channels to Disjount Groups of Discs. 



Again we let one processor serv^ J programs in a multiprocessing 
fashion. However, after a program burst terminates we assume that the program 
requires information frcm one of a group of Discs, each group being accessed 
through its own dedicated (single) channel. For the moment we discuss only 
the two-channel case. The configuration is as shown below. 



Under the assumptions (i) that each program that leaves the CPU 
stage (burst terminates) reports at random — with appropriate probabilities. 



(ii) that all burst time distributions are exponential, we can analyze the 
system by means of the Gordon-Newel.1 cyclic queueing model, [ 5 ] - It turns 
out that several relevant measures of system performance can be explicitly 
written down in parametric form for the present situation; one does not need 
the more comprehensive methods of Buzen [ 1 ] . 

(a) The Long-Run CPU Idleness Probability. 





a 



1 



and not necessarily equal to a — to Channel 1 and Channel 2, and 



J+l J+l 
Po - Pi 



P{CPU idle) = (1-p )(l-p ) 




(3.8) 
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where 



^ a i _ [Channel i (disc) Burst Time! x [Probability Channel i Selected] 

p i u . CPU Burst Time 

1 

i = 1,2, provided ^ p 
If p^^ = p 2 , then 



P{CPU idle | p^=p 2 =p } = 



(l-p) 2 (J+l)p^ 



1 - p J+1 [l+(J+l)(l-p)] 



(3 



(b) Long-Run Channel 1 Idleness Probability. 



P{ Channel 1 idle} = (1-pp (1-P 2 +1 ) 



P 2 ' P 1 



, x J+2 v J+2 

p 2 - p i+( i - p 2 ) p i -(i-p 1 )p 2 



(3 



A corresponding expression for Channel 2 is obtained by simply inter- 
changing and in the above formula. 



(c) Long-Run Expected Number at Channel, and at CPU, Stages. 

It can be shown that the expected number at the channel stage, both 
enqueued and in process of device access, is given by the expression 



E(Number at channels} = 



(1-P 2 ) (!"P 1 ) 



, n J+2 /n x J+2 

P 2 ”P j + (l~P 2 )Pi " (1“P^)P 2 



(i-(j+dp^+jp ^ +1 

'9 L — L - 

2 ( d-P 2 ) 2 



" P 



? Jl-(J+l)p^+Jp^ +1 
1 ( (1-pp 2 , 



(3 



Of course 



E[number at CPU] = J - E[number at Channels]. 

Note that the above formulas do not give the individual occupancies 
(queues plus those in service) at the separate channels, but only the sum 
of those occupancies. 



.9) 



. 10 ) 



. 11 ) 
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(d) Long-Run Expected Number at Channel 1. 



It can be shown that the expected number of programs at Channel 1 
is given by the expression 



E [number at Channel 1] = 



(i-p 1 ) (p 2 -p P 



. s J+2 .. , J+2 

p 2 - p i + ( 1 “ p o) p i “(HJp, 



2 1 



12 L 



;i-(j+i)p^+j p ^ +1 j 

i ] 



■ p i p 2 : 



jl-(J+l) ( P1 /P 2 ) J +J( P1 /P 2 ) J+1 p 

r (i - p i /p 2 )T i 



( 3 . 



The expected number present at Channel 2 can be obtained by reversing 
and p ? in this formula. 

Note that this expression and the one given previously may not describe 
the (expected) number present if the latter number is obtained from obser- 
vations made at special instants of time. For instance, if the Channel 
occupancy is counted just before each new arrival to its queue occurs 
one would anticipate an average lower than that resulting from counts 
made just after each new arrival: think of the case J = 1, for instance. 

Consequently the formulas in (c) and (d) may require adjustment in order 
account for sampling techniques actually used. As they stand, they 
describe the expected channel occupancy when the channel is observed 
either continuously through time, or at uniformly random time points. 
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Configuration 3: One Processor, c Channels to Disjoint Groups of Discs. 

The figure below depicts a very common multiprogramming computer 
configuration at least in an approximate manner. 




A program or job experiences a burst of mean duration X at the CPU and 
then selects a Channel to a group of devices; Channel i is selected with 
probability cu, independently of ail previous events. Thus each Channel 
has its own queue, and the CPU also has a queue. The service rate on Channel 
i is iK. There are assumed to be J jobs simultaneously in the system, 
and we study the process when it is in the equilibrium state. 

Analysis : The Gordon-Newell Cyclic Queue Model. 

In [5] Gordon and Newell present a general solution to the multi- 
dimensional Markov process describing the above setup. Letting N_^ be a 
random variable denoting the number present at Channel i in equilibrium we 
can show that for the present configuration 



where 



p(n n ,n 0 , . . . ,n ) P{N_ n_,N_ n^,...,N n } 

1 z c 1 1 z Z cc 

n l n 2 n c 

= K( J)p p . . . p c , n + n + ... + n * J. 

12c 1 z c 

K(J) is determined by normalization, i.e. by the condition that 



(3.13) 
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( 3 . ] 4 ) 



y ••• l p (n_ , n _ j . . • , n ) 1. 

n-+n 0 +. . .+n £J 
12 c 

We shall derive an easily computed explicit formula for K(J) in the present 
case. Let 

Aa. 

P i = ^ , 1 = 1,2, . . . ,c (3.15) 

l 

the traffic intensity parameter for the single-server queue at Channel i. 

Recall that if Channel i were confronted by a Poisson arrival rate of Acx, 

1 

then the equilibrium distribution is geometric (if p^ < 1), and indeed if 
J becomes very large, so that the CPU is constantly busy, then it is just 
such an equilibrium distribution that is seen. In fact, we can write 

n n n 

^ ^ ^ ^ ^ (i-p >p (i-p >p . . . (i-p >p c 

P{N =n ,N -n , . . . ,N =n |n +N + . . .+N = — — 2 — — (3.16) 

1122 c c 1 2 c P{N 1+ N 0 +...+N 

12 c 

/V 

where N is the state variable of an unlimited Poisson arrival queue, and 
note that 

(l-p )(l-p )...(l-p ) n n n 

p(n ,n , . . . ,n ) = — — p p p . (3.17) 

1 Z C ^ ^ - 12 c 

P{N 1 +N„+. . .+N ^J} 

1 2 c 

Thus all that is necessary to find the normalizing constant is to find the 

distribution of the sum N- + N n + • . . N = N. This step is facilitated by 

12 c 

taking generating functions: 



N . 00 n . n . 

E[z X ] = (l-p > i z y. 1 = 

n . =0 
1 



I-P, 



i 1-zp. 

l 



and thence, by the convolution theorem 



~ c l-p . 

E|z ‘) ■ " 8(z) 
1=1 1 1 



(3.18) 



(3.19) 
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Now suppose all are numerically different, as will often be true. It is 

convenient to carry out a partial fractions development as follows: rewrite 

g(z) as 

g(z) = l (3 - 20) 

i=l 1 P i Z 



and then determine a^ as follows: multiply g(z) by 1 - p^z and let 
z ->• p^. From the product form for g(z) we obtain after cancelling the 
term going to zero, 



n (1-p.) 

i=l 



(i) 



= a . 
1 



(3.21) 



j=l 



(1-P^/P i ) 



where the right-hand side comes from cancelling off the denominator 1 - p.z 

C (i) th 

term against the numerator of (l-p.z)g(z); II means the i — factor 

i=l 

is omitted from the product. Thus we finally have g(z) completely deter- 
mined, and a term-by-term geometric expansion gives the probability distribution 
of N: 



P{N=n} = l a.p? 

i=i 1 1 



(3.22) 



Next 



P{nWi} = l 



la. I = £ a 

i=l 1 n'=0 1 i=l 1 



[l-(p.) n+1 ] 



(3.23) 



and we can put n = J to obtain the denominator of the expression for 

p(n^, . . . , n c ) , and hence the normalizing constant. The explicit expression is 

thus 



12 



p(n ,n , 



’V = 



c 1-p. 

I ' 1 

i=l 



J+l> 



1-P, 



r 

)j=i 



(i) 



p i 



P .-P . 

i J. 



-1 



n- n n 

12 c 

P. P 0 • - -P 
12 c 



(3 



n_ + + . . . + n ^ J . 

12 c 



Several formulas for useful operational measures can easily be derived, 

e.g. from the observation that the d.f. of N = N- + . . . + N is the same as 

1 c 

rw /V Arf 

that of + . . . + = N, given N ^ J. For instance, the probability that 

the CPU is idle is the probability that N = J, so all jobs are at the 
Channel stage; now in order to obtain a formula, ™r ite 

c 



P(N=J } = l a.p. 

1 = 1 



(3 



and then 



r J 

> a.p. 

L* I -i 



p{n=j} = p{n=j|n^j}' = 



i=l 



1 1 



l a . p . 
i=l 1 1 



. J+l 
1-P • 



Jx " -rfr 



i- 1 



j+i 

p i 



. . i 1~P . 
i=l l 



(3 



In summary 



(a) 



Pq(c, J) 



r J 

> a.p . 

i=l 1 1 



1 - l a. 

i=l 1 1_ 



J+l 
P . 



(3 



is the long-run fraction of time the CPU is idle in a c-Channel system with 
J jobs in core. Here 



-24) 



.25) 



.26) 



• 27) 
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a. 

1 



(3.28) 



n (1 -p.) 

i=l 



n 

U=i 



(i) 



P . 






where JI^ represents the product with the term j = i omitted. 



(b) The total Turnaround Time for servicing k programs equals the Total CPU 
Time, divided by 1 - p^(c,J). 

(c) The system under study is closed (actually, completed programs are assumed 
to leave and then to be instantly replaced) , and so a long-run steady state is 
reached. For such a Markovian system we know that a condition of detailed 
balance exists: 

P{CPU BusylXp^ = P{Channel i Busy}p^. (3.29) 

To explain this intuitively, suppose we run the system for a long time, t_. 

Then the CPU is busy (not idle) for a time approximately equal to 

_t • P(CPU Busy} = t_[l- P() (c,J)] ; (3.30) 

the latter is a consequence of renewal theory arguments; see Cox [2]. Now 
at any moment during a CPU busy time a departure occurs with approximate 
probability Adt, and that departure represents a program destined for 
Channel i with probability a^. Consequently the long-run expected number 
of programs that flew towards Channel i is 

t P{CPU Busy }Ao^ 

But the long-run expected number of programs that flow from Channel i to the 
CPU is _t P{Channel i Busy}p^, for all programs that leave Channel i go 
back to the CPU. And we must have balance: 
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Number of Programs from CPU to Channel i = 

Number of Programs from Channel i to CPU 



in the long run, else a buildup will occur at one point or the other. Thus 



_t P{CPU BusyjXa^ = t_ P{Channel i Busy}y^ 



or 



_t P{ Channel i Bus^ Xa^ 

V P{CPU Busy} = ~^7 = P i 



( 3 . 31 ) 



or, approximately. 



Long-run Busy Time for Channel i ^ ^ a i _ , 

Long-run Busy Time for CPU P i 

This approximate equality allows us to estimate p , the input to our evalua- 
tion equation, from monitor data that records total CPU and Channel times. 

We shall put it to use shortly. 



(d) The expected number of programs at the Channel stage can be calculated 
as follows (note that N here denotes the total of all jobs in residence at 
the Channel stage in the long run; so J - N is the number at the CPU): 



N 

l nP{N=n} 
n=0 



J c 

l n l a p? 
n=0 i=l 

J+l 

c 

1 - l a. f- 

i=l 1 1_P i 



c 1-p . (1+J-p.J) 

I a.p. T^j — 

^ i K i (l-p_. 



J+l 

c p. 



1- I a.f- 

• It 1 1“P • 



i=l 
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A similar formula is available for the expected number present at each of the 
i Channels. 

The above formulas may easily be evaluated numerically. Computer 
programs for so doing have been written in FORTRAN and run on the IBM 360/67 
system at the Naval Postgraduate School. In addition, Professor Neagle Forrest 
has programmed the two-channel formulas for the HP-65 hand-held calculator. 
Extension of this latter program to more complex and realistic configurations 
would allow portable use of the models by sizing teams. 
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4 . Fitting the Models to Measurement Data. 



Next I embark upon a discussion of the manner in which some actual 
program material, constructed during the fall of 1974 by Mr. B. Wallack of JTSA, 
behaved (a) when submitted to the Honeywell 6000, being allowed 128K of core, 
and (b) was estimated to behave by the simple analytic (Markov) models that 
have been developed. This discussion is concerned with the first of our 
attempts to validate, or at least test the consistency and perhaps the useful- 
ness of, our simple models. 

A. The Program Material. 

Program material submitted to the H6000 consisted of 40 programs that 
required CPU processing alternating with disc file accesses. Each program was 
so constructed that all CPU "burst” lengths were identical, but burst lengths 
varied between programs. The set of programs was submitted as a batch to the 
H6000 and the following items were recorded. 

(1) Total Job (Problem Program) CPU Time. 

(2) Total System CPU Time. 

(3) Total Disc Time (Single Channel Case). 

(4) Total Channel Time for Each Channel (Two Channels to Two Discs Case) . 

(5) Average CPU Queue (including item being processed) . 

(6) Average Channel A Queue (including item being processed; apparently 
the queue was examined just prior to each moment a job joined the channel 
queue) . 

(7) Average Channel B Queue (same as (6) above). 

(8) Total Turnaround Time (to finish all 40 jobs). 

(9) Total CPU Idle Time. 

(10) Average Number of Programs in Core (later, a "core map" describing the 
number of programs in core at 30 second intervals became available) . 
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B. Fitting the Models. 

It is possible to fit the models to the data furnished by making use 
of the measured data described. I will describe such fits, and comment upon 
them. 

First note that formulas for CPU idleness probability use as basic 
parameter the ratio 

1 

_X _ y_ Expected Disc or I/O burst time 
y JL Expected CPU burst time 

X 

Under the assumption that CPU and I/O bursts essentially alternate (certainly 
not always accurate) there are very nearly as many CPU bursts as I/O bursts, 
so 

A ~ Total Disc Time 
y ~ Total CPU Time * 

fo** present purposes we have added items (1) and (2) to obtain Total CPU 
Time, effectively lowering the CPU processing rate to account for overhead. 

A more sophisticated (but still simple) model has been constructed for over- 
head; we do not attempt to fit it here. 

Next observe that our formulas for CPU idleness pretend that a fixed 
number J, of programs is in core. This is literally untrue, but we shall 
set the average of the measured number in core equal to J, later examining 
the effect of this step (better possibilities suggest themselves). 

Finally, we will calculate CPU utilization via our formulas, and utilize 
the latter to estimate total turnaround time for the 40-program job stream. 
Other comparisons will also be made. 
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C. Actual Cases. 



Case 1: One CPU, One Channel, One Device 



Job CPU Time 



= 0.9830 



System CPU Time 



= 0.0348 



Disc Time 



0.8407 



Average Number of Jobs in Core =3.1 



It follows that we estimate 



y 



0.9830 + 0.0348 



0.8507 



= 0.836 



We may use the basic formula (3.2) and approximate relationship (3. 31, a) to 
estimate CPU idleness probability. The value obtained is 



Total measured turnaround time reported was 1.41; the predicted result 
was about 12% below the actual. This quality of agreement is encouraging, 
considering the discrepancies between the model and the actual program material. 
It is worthwhile to search for explanation of the discrepancy, however. 
Sensitivity to Core Occupancy, J . It is easily possible to compute the effect 
of various core occupancies; we do so for only two additional values: 



P 0 = 0.181 



Total (expected) turnaround time is then estimated by dividing total CPU 



time by CPU utilization: 



Estimated Expected Turnaround Time = 



0.9830 J- 0.0348 
1-0.181 
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d 2 = 0.275, 



and 



d. = 0.136 
4 



1 0178 

Expected turnaround time, J = 2 = = 



Expected turnaround time, J = 4 = 




If we graph expected turnaround time as a function of J the following type 
of curve results: 



Consequently if we have the probability distribution of J, and average first, 
and use the latter average in d T to estimate turnaround time we obtain a 
result that is smaller than that obtained by averaging turnaround time, condi- 
tional on J. Thus, it may be that careful attention to the core occupancy 
stochastic process will deliver more accurate predictions of turn around time. 

In order to study this effect further a "core map" at 30 second intervals 
was made by B. Wallack. This allowed empirical development of the distribution 
of core content for the data and configuration under study here. We find 
(letting p(J) denote the long-run probability that J programs are in core 
simultaneously) the following numerical values. 

Number of Programs in Core (J) Frequency (estimated p(J)) 



Expect . 
Turnaround 




12 3 



J 



1 



0.085 



2 



0.235 



3 



0.268 



4 



0.386 



5 



0.026 
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Under the apparently valid supposition that number of programs in core (core 
occupancy) changes slowly as compared to "service,” i.e. CPU and I/O activities, 
we can argue that for fraction of time p(J) the output rate is essentially 
^[l“d ] , so expected turnaround or throughput time requires the average or 
expectation 



E[ V 




p 



J 



where k = number of programs, and b = average CPU time 



experiment essentially estimates 



kb 

A 



total CPU time. 



data in question we then computed 



(l-d ) 1 for J 



1 , 



J 

1 

2 

3 

A 

5 



1.8360 

1.3807 

1.2305 

1.1566 

1.1132 



per program. The JTSA 
Now for the particular 
.. .,5: 



Weighting the above by the estimated p T 's, and finally multiplying by total 
CPU time yields an estimated turnaround time of 1.31, which is within 7% of 
what was actually observed, although still low. 

Comments . 

Variation in core occupancy may account for the observed discrepancy 
between simple formula (constant occupancy) turnaround estimates and actual 
measurements. Required are (a) methods for estimating the error of our turn- 
around estimate, since the latter is based on experience with finitely many 
programs, (b) a model for predicting the core occupancy distribution, (p(J)}, 
when core size is changed, (c) checks of results when CPU burst times and 
Disc times are not exponentially distributed. 
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Case 2: One CPU, Two Channels to Two Devices, Programs Utilize One 

Device Exclusively. 

Programs were so written as to make use of just one device; this demand 
pattern seems to make the model’s Markovian-random demand structure less 
applicable. Results of model fitting (assuming constant J) are as follows. 
Measured Total CPU Time = 1.3044 

Measured Total Channel A Time = 0.4089 
Measured Total Channel B Time = 0.4425 

P x = 0.3135, p 2 = 0.3392, J = 2.9 

(We use * to denote an estimate from data available.) If these estimates 
are used to obtain 

Measured Model Output 



p o 


0.072 


0.0136 


Turnaround 


1.32 


1.41 


CPU Queue 


1.97 


2.19 



Although the model-predicted CPU idleness differs from that measured by a 
factor of five, the turnaround times are reasonably close. 
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Case 3: One CPU, Two Channels to Three Devices; Programs Utilize One 

Device Exclusively. 

The system configuration in this case allows programs to access two 
disc devices by means of one channel, and another disc by means of a second 
channel. Core size was apparently increased in this exercise also, for the 
average core occupancy was slightly over six jobs. Results obtained from 
measurement and model fitting are as follows. 



Measured 


Total 


CPU Time 


1.1429 


Measured 


Total 


Channel A (16) Time = 


0.6012 


Measured 


Total 


Channel B (8) Time = 


0.2489 


P 1 = 0.5260, 




P 2 = 0.3392, 


J = 6.16 


these estimates 


are used as input we obtain 








Measured 


Model Output 


p^(CPU Idleness) 




0.0246 


0.0122 


T (Turnaround) 




1.18 


1.16 


CPU Queue 
(expected) 




4.86 


4.71 


Device Queue 
(expected) 




1.30 


1.29 



Although the assumptions of the model are not well satisfied for this particular 
workload, the model and measurements generally agree well, although predicted 
idleness probability is lower than that measured by a factor of two. 

Further exercises are being conducted and analyzed, and the results will 
be reported as they become available. In general, there has been reasonable 
agreement between model-predicted results and the measurements made on Honeywell 
equipment that have been furnished to me. 
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5. Statistical Considerations in Fitting Computer Models to Data . 



The previous section described the quality of the fit obtained when 
the simple Markovian computer model was fitted to actual (Honeywell) workload 
data. The purpose of this section is to consider some of the problems of 
statistical estimation that arise when considering model fitting and assess- 
ment . 

A. Inferences from Measurements: Sources of Error. 

A-l. Sampling Error. 

In order to obtain measurements of system performance, strings of 
k = 40 programs were run through each actual computer configuration. It is 
a truism that results obtained from a particular set of 40 programs would not 
be exactly reproduced if other sets of similar jobs were run. 

It is of interest to use the output data actually obtained from a given 
sample to set rough confidence limits on an unknown expected turnaround time. 

In order to do this, the following steps were taken 

(i) Mr. Wallack of JTSA furnished me with the clock times at which 
each of the 40 jobs left the system. These were naturally arranged in 
increasing order; differences between successive outputs were found by subtrac- 
tion . 

(ii) The time differences between successive system departures were 
examined statistically. In particular frequency plots were made of the dis- 
tribution of the time between successive departures, and moment estimators 
were computed. 

(iii) Estimates of mean inter-departure time and standard derivation 
of inter-departure time were in reasonably close numerical agreement (standard 
deviation slightly smaller than mean), which indicates that program exits 
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occur at nearly exponentially distributed time intervals. Such is what one 

anticipate if a "thinning" mechanism is in effect: wi^h small probability 

0 a program exits the system after a CPU burst (to be replaced by one 

enqueued) while with probability 1 - 0 it approaches a channel, later to 

return to the CPU. If exit events are independent, i.e. Bernoulli trials, 

then under our Markov assumptions actual departures should be close to 

exponentially distributed. Assuming equality of mean and standard deviation, 

then, we conclude that total turnaround time, T, is approximately gamma 

distributed, on the basis of which deduction confidence limits are obtainable. 

In fact, total observed turnaround time estimates E[T], expected turnaround 

1/2 

time, and /k x total observed turnaround estimates [Var[T]l . Since k = 40 
is large enough to justify a normal approximation, we state that, with approxi- 
mately 95% confidence, E[T] lies in the interval [(observed turnaround) 

2 

(1 ± — )] = [(observed turnaround) 0 . 68 , (observed turnaround) (1 . 32) ] . 

/k 

On the basis of the above error assessment it appears that a sample 
of 40 jobs is sufficient tc estimate E[T] with an error of about ±30%; 
quadrupling the batch size will reduce this error to ± 15%, with 95% 
confidence . 

It would seem to be worthwhile to keep the above figures in mind when 
models are being assessed — and, for that matter, when systems are being sized 
simply by running representative experimental programs and judgmentally inter- 
preting the results. Although there are alternatives to the error assessment 
presented, the latter is simple and requires only the total turnaround time 
statistic. I plan to study more complex, and possibly more accurate, alterna- 
tives in the future. 
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Turn now to a consideration of other possible causes of real or apparent 
model error. 

A-2. Core Occupancy Tail-off at End of Run. 

Observe that if we knew p^, for i = l,2,...,c, and core occupancy, 

J, remained constant, and further that our models’ assumptions (Markovian- 
exponential) are well satisfied, then the long-run formulas will predict E[T] 
accurately provided the number of programs, k, is "large 11 enough so that 
transients at the beginning and end of a run are insignificant. None of the 
abov° assumptions are strictly true, and an examination of core occupancy data 
("core maps") shows that there is some tailing off of occupancy as the end of 
a run is approached. Just how serious this effect is upon our estimates is 
under investigation at JTSA. Such tail off should tend to artificially prolong 
experimental runs. 

A-3. Core Occupancy: Slow Variations. 

The assumption that the number of programs in core is fixed and equal 
to the expected (average) number of jobs in core is an oversimplification, 
and tends to produce a constant bias, as has been remarked. A model that 
addresses this problem is under construction. 

A-4. Program Material Statistics. 

If CPU and Channel-Device burst times do not resemble independent 
exponential random variables then systematic prediction errors are likely to 
occur. In particular, if CPU bursts tend to be hyper-exponential (skewed, 
long-tailed) appearing, as may be the case when mixtures of programs are 
present, then the Markcv model understates CPU idleness and turnaround time; 
see Gaver [2], and Gaver and Shedler [4]. Before a more refined model 
can be fitted, however, further measurements must be made and interpreted. 
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6. A Markovian Model that Includes Overhead. 



The purpose of this section is to derive a simple model for a 
multiprogramming computer system that explicitly includes some of the 
possible effects of ’’overhead” actions by the CPU. It will be assumed that 
at the termination of every activity "burst,” whether it be CPU or I/O, the 
CPU enters an "overhead" state, residing there for a period long enough to 
carry out activity so described. 

In order to keep the analysis and formulas simple the derivations 
will be carried out for a Markovian system. Our objective is to see, at 
least qualitatively, how overhead decreases overall system productivity. 

The present results, although based on oversimplified assumptions, are per- 
haps a degree more realistic than is the simple ad hoc procedure of reducing 
CPU production rate, X, by an empirically derived factor. 

A. Derivation . 

Let N(t) denote the number of programs at (waiting, and being served 
by) the CPU at time t. Let 



Pj(t) = P{N(t) 
QjCt) = P{N(t) 



j, and CPU doing problem program computing), 
j, and CPU performing overhead). 



Let v ^ be the expected duration of an overhead activity ( any overhead 
activity, according to our present simple setup). Furthermore, assume that 
at the termination of any problem program work, be it at CPU or I/O, the CPU 
is immediately preempted for a random, exponentially distributed time of mean 
duration v \ 

If j programs are at the CPU, let X denote the CPU program burst com- 
pletion rate, and let p^ be the corresponding rate for I/O. Later on we can 
fill in specific functions for X and p.; at present it seems unnecessary. 
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Now write down the basic probability equations for P^(t) and Q^(t): 

P.(t+dt) = P.(t)[l-(A.+y.)dt] + vQ.(t). (6.1) 

J J J J J 

the latter following because if the system is in CPU-active states j+1 or 
j-1, and CPU or I/O completes, then the system can only go into CPU-overhead 
state j, and thence to active state j. Next 



QjCt+dt) = Q (t)[l-vdt] + p j+1 (t) x j+1 dt + (6.2) 

the above explanation covers the latter equation. 

Conversion to differential equations, and thence to balance equations, 
dP . dQ . 

is routine (set = 0, =0 to get balance). The balance equation 

results are: 



p . (A ,+y . ) = vq., vq. = A p + y. p. 

J J J J J J+l J+l J-1 J-1 



(6.3) 



Therefore, equating vq_. terms gives 



P.a.-H,.) - » j+1 p J+1 + “j-iPj-r 



(6. A) 



Start with 



X 0 ‘ 0 = U -1 



Vo ' Vi 



(6.5) 



and repeatedly solve: 



P 1 P 0 X, 



y o P 1 Vl 

P j P 0 X 1 x 2 ... X. 



( 6 . 6 ) 



and 



A ,+y . 



q, = P 



p o y i ••• y i-i - Ax+y - 



f - J- - J ) = p - u - . 1 r-J— J.”) 

j 1 v J p 0 X 2 ... X 1 v > 



(6.7) 
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Finally, 



oo 



l (p.+q.) = l. (6.8) 

j=0 J J 



B. Example . 

Suppose J jobs multiprogram and make use of a single channel to 
various devices. Therefore, X^ = X, y. = y, and 



P, = P 



© 



o v \ 



Oj ■ -o© 



(6.9) 



from (6.6) and (6.7). Summing up from j = 0 to J gives, with the aid 
of (6.8), 



Po = 






( 6 . 10 ) 



0 v+X+y J+l 

*-© 

The long-run fraction of time during which the CPU is concerned with overhead 



activity is 



i - p 0 (^) i © J 

j=o j=o 



A+p 

A+p+v 



( 6 . 11 ) 



29 



7. Conflict on Channels Leading to Memory Devices. 



A. The Setup: Two Channels to Several Equi-Demanded Discs 

Consider the following configuration, describing a system with a 
CPU, several memory devices (e.g., discs), and two channels by way of 
which the devices are accessed. 




In conformity with Markov assumptions let X be the CPU service rate, 
so X 1 is an expected CPU burst, and let y ^ be the expected device 
burst. 

It will be assumed first here that at the termination of any CPU 
burst the program is equally likely to proceed to any device . There may 
well be an attempt to ensure this type of behavior by shifting filed 
information around to equalize device appeals. Other assumptions 
(unequal probabilities) are more difficult to handle, and must wait. 

They can be handled in an analytical framework, but more equations must 
be solved. 

Further assume that, so long as there is no conflict for a 
particular device T s information, the rate of service is — where n 
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now denotes the number of programs in the channel-device stage — 

if n = 0 

if n = 1 ( / . 1 ) 

if n ^ 2 . 

If, on the other hand, a request comes from the CPU for information from 
the same disc or one already in use, and reaches the head of the device 
line, then 

(O if n = 0 

u „- , (? - 2 > 

( y if n ^ 1. 

If there are D devices in all, the chance of a conflict is assumed here 
to be D ^ ; this assumption will be changed if necessary. 

Assume also that service is in arrival order, so that if a channel 
is connected to device i in order to service a particular program, and 
if a program immediately behind the one being served also requires the 
disc in question, then effectively the second channel is blocked. 

B* Probability Model 

Let (a) P^(t) denote the probability that n programs are 

present at the device stage (waiting and in service) and both channels 
are available (no blocking) , while (b) B^(t) denotes the probability 

of the same event, save for the difference that one channel is blocked. 

Let X be the CPU burst rate (A ^ = expected CPU burst time) , 
and y ^ be the expected time for which a disc is continuously engaged 
by a single program. We let the probability that a program is blocked 
be 6 (B = -jj, D being the number of discs), and 3 = 1-3- 
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B-l Probability Equations 



(1) P Q (t+dt) = P Q (t) [1-Adt ] + P 1 (t)ydt + o(dt), (7.3) 

for no blocking is possible when n = 0; the first term on the right 
hand side (RHS) represents the probability that the number of programs 
at the device-wait stage is 0, and no new arrival occurs in (t ,t+dt) , 
while the second is the probability that 1 was present, occupying a 
channel, but terminated in (t,t+dt). 

(2) P (t+dt) = P 1 (t)[l-(A+y)dt] + P Q (t) Adt + P 2 (t)2ydt + B 2 (t)ydt 

+ o (dt) . (7.4) 

The only term requiring remark is the last on (RHS) : B 2 (t) is the 

probability that 2 are present, but only one channel is in use (blocking) 
but in (t,t+dt) the one in service leaves, and either channel is now open 

(3) P 2 (t+dt) = P 2 (t) [l-(A+2y)dt] + P (t)A3dt + P 3 (t)2yPdt 

+ B 3 (t)yBdt + o(dt). (7.3) 

The first RHS term represents no change from 2. The second term 

represents the probability of 1 present and a non-b locked arrival. The 

third term means that 3 are present, 2 of which are in service; a 

rd 

departure occurs, and the 3 — is non-blocking. The fourth term means 
3 present, one channel blocked hence one in service, a departure occurs 
in (t,t+dt) and the next two do not require the same disc. 

In general, for n > 2 an arrival doesn’t enter service, so 
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(4) P n (t+dt) = P n (t)[l-(X+2y)dt] + P r _ 1 ( t) Adt + P n+1 ( t) 2p6dt 

+ B n+1 (t)ygdt + o(dt) (7.6) 

Turn now to equations for B (t) . There can be no blocking for 
n = 0 or 1, so Bg(t) = B^(t) = 0. We omit writing o(dt) . 

(5) B 2 (t+dt) = B 2 (t)[l-(X+y)dt] + P 1 (t)X8dt + B^OyBdt + P 3 (t)2y8dt. (7.7) 

The first term represents no change (only one is in service) . The second 
represents 1 in service and the arrival of a program requiring the same 
disc. The third represents 3 present, 1 in service and one channel 
blocked; the departure of the latter finds that the next in line is blocked. 

The fourth means 3 present, 2 in service; one departs, and the third is 
blocked. 

For n ^ 3, 

B n (t+dt) = B (t) [l-(X+y)dt] + B 1 (t)Xdt + B n+1 (t)p8dt 

+ p n+1 (t)2vBdt (7.8) 

To derive differential equations, subtract P (t) or B (t) from 

n n 

the RHS as appropriate, divide by dt and let dt 0. Balance equations 
follow by setting derivatives equal to zero. 

U') *P 0 = PP X 

(2') (A+p)p 1 = Xp Q + 2pp 2 + yb 2 

(3') (A+2y)p 2 - A6 Pl + 2y3p 3 + pBbj (7.9) 
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(4’) 



(A+2y)p n = Ap n _ 1 + 2p3p n+1 + uSb^ 



(4', a) 2ypj = Ap^ 



for n = J 



(5 f ) (A+y)t> 2 = + 2y6p 3 

(6 1 ) U+y)b n = Ab n-1 + y3b n+1 + 2ySp n+1 , J-l ^ n ^ 3 
(6\a) yb J = Ab J _ 1 

Special Case . Let the number of programs in the system be J = 2. Then 
we can solve ( 1 1 ) , (2 T ), (3 T ), and (5 1 ) simultaneously: 




yield 



1 





The probability of CPU idleness is b^ + p^, so 



Expected CPU utilization = 




( 7 . 



Numerical Example , Program material synthesized by B. Wallack of JTSA 
yielded the values 



Our formula for J = 2 shows that if 6 = — = ,, _ 

D If of disc units 



so if 



Expected CPU utilization 



1.83 



1.83 + 0.689 




D = 2, Expected 
D = 4, Expected 
D = Expected 



CPU utilization 
CPU utilization 
CPU utilization 



0.78, while if 
0.81, and if 
0.84 



In order to find CPU idleness probability in the general case, 

we must find p 4- b . This can be done by solving equations (l 1 )-^ 1 ) 

J 

. + £ b . = 1. There 

3=0 J 3=0 J 

are in all (J41) - 1 - Cl 4 -l-2) = 2J linear equations to be solved; any 
available package program for linear equation solutions should be 
applicable . 

Another appealing possibility for obtaining a solution is to use 



simultaneously, subject to the condition that l p 



an iterative procedure of successive corrections. 



C. Approximations 



The simplicity of the Markovian channel problem suggests that a 
good, explicit, approximate solution may be available. One thought is to 
examine the Disc stage at times when blocking can occur, i.e. when £ 2 
programs are awaiting Discs (of course, one may be blocked). 

Suppose, then, that the Disc stage is "flooded," i.e. that the CPU 
is fast enough to keep the latter constantly working. Let P(t) denote 
the probability that the channels are both busy at t, and B(t) denote 
the probability that one is blocked; clearly P(t) + B(t) =1 when 
flooding occurs. Now 

P(t+dt) = P(t)[l-2ydt] + P(t)2yBdt + B(t)yBdt (7.11) 

which leads to 

^ = -2m P(t) + 2\il P(t) + y3 B(t) 

= -2m 3 P(t) + MB[1-P(t)]. (7.12) 

To find the steady state probability 



p = lim P(t) 

t-x» 



set the derivative equal to zero and solve: 

B 



P = 



i - ± 
1-3 D 



23+3 



1+3 



1+ £ 



D-l 

D+l 



(7.13) 



and 



b = lim B(t) = 1 - p = 



t-x» 



23 = _2_ 
1+3 D+l 



(7.14) 
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Consequently the steady state output rate from the "flooded" 



channels is 



2pp + ub = 2p(|^) 



+ u 



2 

D+l 



= 2p 



D 

D+l ’ 



and the effective output rate per channel is 



p = 



p 



D 

D+l 



Now approximate the Disc stage service process as follows: 



P 



1 



P 



P = 
n 



2p 



n ^ 2 



and treat the delays at the Disc stage queue as simple Markovian, with 

arrivals X as before and the above service rate. Let d denote the 

n 

steady state probability that n programs are present at Disc Stage 

(d ^ p + b ). If this assumption is made, 
n n n 



so 



Since 



X n x X ... X 
, , 0 12 n-1 

d - 

n 0 p., P„ P„ 



1 2 3 



P 



n 



d„ = 



a A 
d o p 



d = d 



d = 



0 ~ 

2pp 



n-1 



n = 1 , 2 , . . . , J 



( 7 . 15 ) 
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Since 



1 = l d = d 

-n n 0 
n=0 



i + £ 

n=l 2 y 



= 



, , X 2m I 

1 + ^i7T3Tl 

2p 



and the probability that all J programs are at the Disc stage is 



d T = 



2u 



J-l 



X ^ J 



1 + 



A | 2yj 



Ml- — 

A/ 

2p 



(7.16) 



then 1 - d = CPU utilization. 



Special Case . Let J = 2. Then 



A # _A 
P 



2p 



d2 ' i - i(i + 43 

M 2y 



and 



1 - d„ = 



1+* 
y 



i + - + - — 

y 2p 



1 + u 

i + * + 1 (V <m 

p 2 V 1 D J 
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which agrees with the answer found by using the correct Markovian analysis 
for this case. It thus seems reasonable to use the approximate result 
dj in order to estimate CPU utilization. 
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